Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Static linking of native libs #44505

Merged
merged 32 commits into from
Dec 7, 2020
Merged

Static linking of native libs #44505

merged 32 commits into from
Dec 7, 2020

Conversation

VSadov
Copy link
Member

@VSadov VSadov commented Nov 11, 2020

Closes:#41299

Statically linking libSystem.Globalization.Native and libSystem.IO.Compression.Native/clrcompression and use overrider mechanism to invoke.

  • Added support for PInvoke overriding mechanism. A static host or embedder can provide a callback to participate in PInvoke resolution,
  • At the time of PInvoke resolution, if there is an override it gets a chance to provide an implementation before normal PInvoke resolution process.
  • libSystem.Globalization.Native no longer relies on "QCall" mechanism.
    Instead there is a default/built-in override, which runs right after the host-provided one. Host-provided override has preference since it runs first.
  • The default overrider can be extended if we add more cases where PInvokes are backed by implementations statically linked into runtime (regardless of single-file or .dll/.so scenarios)
  • The signature of the overrider callback is:
const void* (PInvokeOverrideFn)(const char* libraryName, const char* entrypointName);

Returning NULL is interpreted as "no-match" case.

  • The overrider is passed to coreclr_initialize as a property named PINVOKE_OVERRIDE. The value is a pointer encoded as a hex-based string value. This is similar to BUNDLE_PROBE.
  • The implementation of an override is up to the overrider.
    In this change the following scheme was used:
    -- Host overrider dispatches on the name of the library (text match) and defers to one of the embedded libraries, if there is a match.
    -- The library dispatches on the native name of the Dll Import and provides a method pointer of there is a match.

@EgorBo
Copy link
Member

EgorBo commented Nov 20, 2020

@VSadov Are you planning to also link System.Security.Cryptography.Native.Openssl statically?
We noticed a small issue - this lib is optional on macOS - it's needed only when you perform AES-GCM/CCM encryption. But it has an __attribute__((constructor)) attribute that invokes abort if OpenSSL is not found. So I wonder how it will work in the static linking scenario - it shouldn't abort immediately if OpenSSL was not found on macOS since most of the stuff is AppleTLS-based there.

@VSadov
Copy link
Member Author

VSadov commented Nov 20, 2020

@EgorBo - I was planning to. It would be one of the last libraries I would look at. Not sure how that would work. Thanks for the heads up!

@akoeplinger
Copy link
Member

I also noticed another __attribute__((constructor)) function in libSystem.Security.Cryptography.Native.Apple.dylib: InitializeAppleCryptoSslShim().

I think we should do what we do for globalization and call these initialize functions from managed code explicitly rather than relying on the magic __attribute__((constructor)) initialization.

@VSadov
Copy link
Member Author

VSadov commented Nov 20, 2020

Considering __attribute__((constructor)) issues, I think perhaps linking these libraries on OSX should be a separate change. If that is possible.

VSadov and others added 2 commits December 5, 2020 12:41
Co-authored-by: Ryan Lucia <ryan@luciaonline.net>
Co-authored-by: Ryan Lucia <ryan@luciaonline.net>
@VSadov
Copy link
Member Author

VSadov commented Dec 7, 2020

runtime (Build iOS x64 Release AllSubsets_Mono) seems to be failing even in a no-change PRs

@VSadov
Copy link
Member Author

VSadov commented Dec 7, 2020

I am going to merge this. Thanks to everyone who helped with this PR!

@VSadov VSadov merged commit 08ad834 into dotnet:master Dec 7, 2020
@VSadov VSadov deleted the libQ branch December 7, 2020 04:08
akoeplinger pushed a commit to mono/mono that referenced this pull request Dec 7, 2020
This PR is a copy of dotnet/runtime#44505

Co-authored-by: VSadov <VSadov@users.noreply.github.com>
monojenkins pushed a commit to monojenkins/mono that referenced this pull request Dec 8, 2020
Superhost PR follow-up

Re: dotnet/runtime#44505 (comment)

For the most part a search/replace change.
akoeplinger pushed a commit to mono/mono that referenced this pull request Dec 9, 2020
…#20633)

Superhost PR follow-up

Re: dotnet/runtime#44505 (comment)

For the most part a search/replace change.

Co-authored-by: VSadov <VSadov@users.noreply.github.com>
@ghost ghost locked as resolved and limited conversation to collaborators Jan 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.